Smooth Boosting and Linear Threshold Learning with Malicious Noise

We describe a PAC algorithm for learning linear threshold functions when some fraction of the examples used for learning are generated and labeled by an omniscient malicious adversary. The algorithm has complexity bounds similar to the classical Perceptron algorithm but can tolerate a substantially higher level of malicious noise than Perceptron and thus may be of signiicant practical interest. At the heart of our algorithm is a new boosting procedure which is guaranteed to generate only distributions which are (optimally) smooth. By using this boosting procedure in conjunction with a noise-tolerant weak learning algorithm, our algorithm can learn successfully despite higher rates of malicious noise than previous approaches.